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1. Introduction 


Let £ be a smooth elliptic operator whose coefficients are 
defined on D X [0, T], where D e E n (Euclidean n-space). 

We consider the problem of the optimal control of systems 
with the formal representation 


W , = ;£W + bu+ £ a . i . 
t 11 


(*) 


satisfying W(x, t) -» 0 as x -» c5D, t S 0, and with control 


u(x,t) = / k(v,x,t)W(v,t)dv 


and cost criterion 


T 


C U (cp,t) = / Jw(x, s)W(y,s)S(x,y, s)dxdyds 


9 


u P 

+ E / /P(x,s)u (x, s)dxds, 


m 

T t 


where | is the formal derivative of the Wiener process z^(t) 
and E^ is the expectation given the control u, and initial con- 
dition cp(-). A precise meaning is given to all terms in the 
sequel. An equation of the form (*) seems like a useful model of 
a variety of noise disturbed objects, but it also arises in the 
following way. Suppose that an object is governed by = ;£H + bu 

and the noise corrupted observations having the Ito differential 
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(for each x e D) dy^(x,t) = dt/nu (v,x, t)H(v, t)dv + dw_^ are 
taken, where the w^ are Wiener processes. Then, the conditional 
expection*" W(x,t) == E(H(x, t) | y(v, s), s ^ t, v e D] has a 

representation in the form (*). In fact, (26) is the relevant 
'Riccati' equation (with a reversed time parameter). The results 
herein concern the first boundary value problem for a single 
* parabolic equation. However, it is clear that the method is 

applicable (and results easily extendible to) the second boundary 
value problem, or to a family of parabolic equations. The latter 
model is quite versatile. For example, we can use a vector para- 
meter process W(x,t) generated by (*) as the input to another 
system (e.g., (<5/(k,-£)Y = W + b' u' ) . We can thus generate the 
'distributed system’ analogs of the ’linear Gauss-Markov’ processes, 
and treat the corresponding average quadratic cost control problem. 

The results are based on the results of [1] which provide 
criterion which guarantee that there is a version of a vector 
parameter process which with probability are (w.p.l.) is continuous 

Ox uiiicxciioiaDzc xu sOrut: yai ox^uxcii jJctx cuuc oefb • WiuflOuo untibt; 

latter results, eq. (l) would lose its intuitive meaning (as would 
stochastic differential equations if the paths were not known to 
be continuous w.p.l.). Previous works concerned with 'random' 
partial differential equations [2], [5], were concerned with the - 
nature of the random solution corresponding to a random, but 


The filtering problem will be treated in a subsequent work. This 
paper is devoted strictly to the control problem. 


3 


smooth, boundary condition. Once the necessary smoothness properties 
of the process W(x, t) are established, much of the analysis is 
similar to the analysis of the corresponding deterministic problem. 
However, to our knowledge the few treatments of the deterministic 
problem (e.g.; see the interesting reference [4]) are essentially 
formal in nature. Most of the proofs are slightly abbreviated. We 
' have chosen to omit the details of several arguments dealing mainly 
with the smoothness properties of potentials and related integrals. 
The arguments are tedious and standard. Some are based on existence 
theorems (Lemma 2.3) and most others use the arguments of [5], 

Chapter 1, Section 3-5. 

In Section 2, some needed results on processes with a vector 
parameter set are given. The proofs of the statements of Lemma 2.1 
are found in [1]. The proofs of the statements of Lemma 2.2 and 
its corollary follow from Lemma 2.1 and the properties of stochastic 
integrals depending on a scalar time parameter. Theorems 3-1 and 
3.2 define the solution of (*) and its basic properties; continuity 
w.p.l, existence of Holder continuous w.p.l. second derivatives 
(with respect to the x^), etc. The optimality and ’ approximation 
in policy space 1 results appear in Section 4. Although we deal with a 
single ’white noise’ input, the results are obviously valid for the 
no more general, ’infinite dimensional’ white noise input of the 


Corollary to Lemma 2.2. 
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by (Djf (•)), i=l,...,r) ([ f ], Theorem 4). 

Under the condition (j&| is the Euclidean norm) 

E|f(y+6)-f(y)| a g |6| 1+P (**) 

for some a > 0, P > 0, there is a Holder"^ continuous (with Holder 
exponent a/p) version of f(-) on R w.p.l. (f 1], Theorem 2). 

Rote that the above implies that if f (y) and the mean square 
derivatives with respect to y , ...,y satisfy (**), then a version 
of f ( • ) has continuous derivatives with respect to y , . . . , y , 
w.p.l. 

An immediate consequence of Lemma 2.1 is 

Lemma 2,2. Let x e D and s,t e [0, T] . Suppose 
R = D X [0, T] satisfies the conditions on R in Lemma 2.1. Let 
z s Wiener process and suppose the function a(x, t, s) satisfies 

T 

x 2 

• I CC (x, t, s)ds < CO 
o 

for each x e D and t e [0, T] and 


"^By Holder continuity (with exponent y~ > 0) we mean that 
| f (y+5)-f (y)| g K(o>)|5|^ where K(co) < °o w.p.l. and t, not de- 
pend on y. In works on partial differential equations, where we 
let y = (x,t), Holder continuity is meant to imply | f (x,t)-f (y, s)| 
K(o>) (| x-y| ^+| t-s| . Since we are not concerned with the specific 
value of y, either form is suitable in this paper. 


6 


2 

/ [a(x,t+A, s)-a(x,t, s)] ds g e(A) 
o 

t (***) 

/ [a(x+8,t, s)-a(x,t,s)] 2 ds ^ e(|5|) 
o 

where e(A) -> 0 as A -> 0. Then ^ the random function 


t 

i|r(x,t) = / a(x,t,s)dz 
o 

has a Holder continuous version on R w. p.l., for some Holder ex- 
ponent x > 0. If , further , the D^a(x,t, s) = 3tt(x, t, s)/dx^ 
satisfy the conditions on a(x, t, s) above, then the random func- 
tions 

t 

(Dn[r(x,t)) =/ D ia (x,t,s)dz s 


can be identified with the mean square derivative of i(r(x, t) with 
respect to x^, and there is a version of \|r(x, t) which is con - 
tinuous and has Holder continuous (some non-random exponent y > 0) 


"I’The details can be deduced from Lemma 2.1. The reader is referred 

to [ 1 ], Example 2 for the details of a similar result. (*-*-*) im- 

t+A t ; 

plies (**) of Lemma 2.1 since E{J a(x+5 t+A, s)dz -J a(x,t, sjdz } 

+ o s o s 

«-f O m 

= (2m-l)’ *5. 3.1 [/ (a(x+S,t+A, s)-a(x, t, s)) ds] which is 
0(|A| 1+P +|6| 1+P ) for some p > 0 and a sufficiently large m. 
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derivatives ( with respect to x.) ® R w.p.l. If the second 
derivatives ( with respect to the of a(x,t,s) satisfy the con- 
ditions on a(x,t,s), then the conclusions hold for the second 
derivatives of \|r(x,t) with respect to the x . 

Corollary . Let i = 1, . . . , be a sequence of independent 

Wiener processes on [0,T] and let the family a (x,t, s) satisfy 




/ ? 
0 ! 


[a^x+Sjt, s)-a(x,t, s)] 2 ds ^ e(|5|) 

2 

[a i (x,t+A,s)-a i (x,t,s)] ds g e(A), 


where e(A) -> 0 as A -» 0. Then w. p.l. there is a Holder con- 


tinuous version of 


t 

iKx,t) = E / a i (x,t,s)dz ig . 
i o 


The remainder of the 
analogs here. 

Define D. 

l 

Uj. = cXj/ cfc and 


statements of Lemma 2.2 have their obvious 


as the differential operator c)/ck^ and let 



■ Let us collect the following assumptions here. Note that Holder 
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continuity on the compact set R is equivalent to uniform Holder 
continuity on R. 

(E-l) Let R = D X [0,1], where D is a bounded and open Borel 
measurable domain. To each point on let there exist a 

neighborhood V and a function on h(-) so that 3D fl V has the 
representation = h(x_^, . ..,x_^ X i+i> • • • t x n ) ^ or some component 
where h(x) has Holder continuous second partial derivatives^. 

(E-2) On R, the coefficients of £ are bounded and Holder con- 
tinuous . 

(E-3) There exists a real number K > 0 so that 


£ a (x,t)| | 1 K Ztf 

1,3 1 J 11 


for any vector £. 

(E-4) Let cr(x, t),D.cr(x,t),D.D ,cr(x,t) be Holder continuous in R 

(E-5) Let a(x, t) and £a(x, t) tend to zero as x -» 8 D in R 

(E-6) Let b(x,t) be Holder continuous on R and k(y,x,t) be 

bounded, measurable and Holder continuous in x, t on R, uniformly 
in y. 


(E-7) Let a..(x, t) have Holder 
R, and d^(x,t) Holder continuous 


continuous second derivatives in 
first derivatives in R. 


+ 


(E-l) implies the condition on R of Lemma 2.1. 
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It will be helpful to collect the following results here. 
They will be used without reference in the sequel. 

Lemma 2.3A ([ 5 ], Theorem 7, Chapter 3 ). Suppose (E-l)- 

(E-3). Let <p(x, 0) have Holder continuous second derivatives in 
D and satisfy cp(x,0) -» 0 and ;&p(x,0) ->0 as x -» 3 d. Let 
<p(ciD,t) = 0. Suppose that f(x,t) is Holder continuous in R and 
tends to zero as x dD. Then there is a continuous unique solu- 
tion to U = + f on R which satisfies the boundary condition 

cp(x,t). U^x, t),DdJ(x, t) and DJXU(x,t) are Holder continuous 

on R and :£U(x,t) -» 0 as x -> dD, t g 0. 

— o — 

Define the Banach space C^ of functions on R which 
satisfy f(x, t) -» 0 as x -» dD and with the norm 



sup_ | f (x,t) | 
x, teR 


sup , 

x, teR I x-y| a +| t-s | a ' 

y, seR 


Let cZ be the sub (Banach) 
2+a 

satisfy j£f(x,t) -» 0 as x -» 


space of C of functions which 
_f a 

dD, and have the norm 


11*11 


2+a 


a 


S IlD.fl! + Z||D.D f|| 

i *, J 


a 


Then, with homogeneous boundary conditions , the equation 

Uj. - a!U = f represents a continuous linear map of C° into 

^2+a’ ( Follows * rom [5 ], Chapter 3,' eqn. 2.21, and the first 
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part of the Lemma. ) 

Lemma 2.3B ([5], Theorem l6, Chapter 3). Suppose (E-l)- 

(E-3). Then there is a Greens function Gfx^x’^t^t' ) for = 

^U. G(x,x* ; t, t 1 ) -> 0, a£ x -> cD for t > t* . D i G(x,x* ;t, t’ ), 

D^D^G(x,x' ‘t } t* ) and G^fx^x' ,* t,t’ ) are continuous in (x,t) on 

D X (t* ,T]. If cr(x, t) is_ continuous in x for each t, then 

a(x,t,t* ) = /G(x,x* * t, t’ )cr(x' ,t' )dx l satisfies (d/dfc-a!)a(x,t, t’ ) = 

cr(x,t) on D fl (t* ,T] and tends to zero as x for t > t* . 

If cr(x,t), and ;£cr(x,t) tend to zero a£ x -» cD and are Holder 

continuous on R, then a(x,t,t' ) satisfies the same conditions for 

(x, t) e D fl [t* ,T]. If. cr(x,t) is_ hounded and measurable, then 
t 

/ dt’a(x, t, t> ) tends to zero as x -> 3 d and is continuous on R. 
o 

(The last statement follows from the arguments concerning potentials 
in [5], Chapter 1, Sections 3-5.) 

Lemma 2.3C ([5], Theorem 17, Chapter 3). Suppose (E-l)- 

(E-3) and (E- 7 ). Then the Greens function for the adjoint operator 
c/dt + £* is 


G*(x,x* ; t,t* ) = G(x’,x;t',t). 

Note that G*(x,x’ Jt, t* ) is defined for t < t’ and that 
(d/dt+;£*)G*(x,x';t,t' ) = 0 on D X [0,t»). 



5 . The Stochastic Partial Differential Equation 

Theorem 3.1. Suppose (E-l)-(E-5). Let cp(x) have Holder 
continuous second derivatives on R and tend to zero as x c£>. 

Let ;&p(x) -> 0 as x -» cD. Then there is a process W(x,t) with 
parameter set R satisfying (z* denotes z(t*)) 

W(x,t) = /G(x,x';t,0)cp(x' )dx* 

t « 

+ / dz’ {/G(x,x* *t,t’ ^(x* f t* )dx* } . 
o 

There is a version of W(x, t) which is continuous on R w.p.l. and 
satisfies W(x,0) = cp(x), W(<£>,t) = 0. D.W(x,t) and D.D.W(x,t) 
are also Holder continuous on R w.p.l. and ^W(x,t) -> 0 as x -x£). 
Also, w.p.l. 


W(x,t) = /G(x,x* ;t, s)W(x* , s)dx* 
t 

+ / dz* /G(x,x’ • t, t' )<t(x* f t 1 )dx* . 
s 


( 2 ) 


For each fixed x in D, W(x,t) has the Ito differential 


dW = ;CW(x,t)dt + or(x,t)dz. 


( 3 ) 


Remark . W(x,t) is not (w.p.l.) differentiable in t. The 


smoothness of cr(x,t) determines the smoothness of W(x, t). Lemma 2.2 
plays a crucial role here. It is not a priori obvious that the last 
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term in (1) has a version which is sufficiently smooth w.p.l. Lemmas 
2.1, 2.2 turn the estimates of the stochastic continuity of (l) and 
the stochastic continuity of its first two mean square derivatives 
(with respect to the x^) into a statement concerning the existence 
of a version for each x, t, so that the actual sample functions are 
smooth. The order of integration in the stochastic integral in (l) 
must he preserved. Also, the theorem implies that the a£W term in 
(3) has a version which is continuous w.p.l. on R. 

It can be shown that the process W(",t). with parameter t, 
is a continuous Markov process with values in a Banach space of func- 
tions which satisfy the appropriate boundary conditions (W(x, t) -> 0 
and ;£W(x, t) -» 0 as x -» 2(D) and have Holder continuous second 
derivatives (for some fixed non-random Holder exponent) . 

Proof. The proof is a consequence of Lemmas 2.2 and 2.3. Let 
p(x,t) be the first term on the right of (1). Then (Lemma. 2. 3) 

= P * p(x t) satisfies the boundary conditions and $3(x,t) -» 0 
as x -> ct) for t 1 0. B(x,t) and jEp(x,t) tend to cp(x) and 
jftp(x), resp. as t -» 0. Write 

a(x,t,t» ) = /G(x,x* ;t,t‘ )o-(x T ,t» )dx' . 

Then for t k t* (t* fixed) a(x,t,t* ), D a(x,t,t' ) and D.D .a(x,t,t' ), 
are Holder continuous on R and each satisfies the conditions on 
a(x t t» ) of Lemma 2.2. Hence, by Lemma 2.2, there is a version of 
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t 

t(x^t) = / dz'a(x,t,t* ) 
o 

which is Holder continuous w.p.l. on R, and which has Holder con- 
tinuous second derivatives with respect to the x^ w.p.l. on R. 
DnK:x,t) and DJ^.i|r(x,t) can he identified with the Holder con- 
tinuous versions (which exist w.p.l. in R) of 

t t 

Dnl^t) = / dz'D a(x,t,t') = / dz'/D.G(x,x’;t,t')cr(x',t')dx» 
o o 1 

t t 

i|r( x ,t) = / dz'D.D a(x,t } t') - / dz'/D.D.G(x,x*;t,t« )o-(x« , t» )dx* , 
d o ^ d o 1 d 


resp. Thus £\[f(x^t) has a version which is Holder continuous w.p.l . } 

t 

and which is clearly a Holder continuous version of / dz* £a(x, t, t* ) . 

o 

Using the continuity of ^i|r(x,t) and the fact that ;£\|f(x,t) -> 0 in 
probability (since ia(x,t,t’ )-» 0) as x -» dD, we have ;£i|r(x,t) -» 0 
as x -» dD. Thus (l) satisfies the required boundary conditions. 

. . A 

nquauion [3 ) follows from the definition of the Ito differential of 

t 

(l) for each fixed x, and the observation that df dz'a(x^t,t' ) = 

t ° 

dza(x,t,t) + dtf dz’a. (x t ,t* ) where a(x t.t) = cr(x t) and 
o 

a t (x,t,t* ) = ^a(x,t,t* ). 

Equation (2) is obviously true w.p.l. for each fixed x,t, s. 

To show that it is true w.p.l. on D x [0,T] X [0,T], note first that 


t 

T|r(x,t,s) = / dz' {/g(x,x t ’ t, s)cr(x' f s*)dx' } 
s 
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can be defined (using previous arguments) to be continuous (as a 

function of x, t, s) on D X [0, T] x [0, T], except for co in some 

null set N. Define \|r(x,t) as the (unique) lim i|r(x,t, s). 

s -» 0 

The limit exists for cn / N, and is a version of the continuous 
(for cd / N) function i|/(x,t) defined previously. Then, for 

® ^ N, 


t(x, t) = *(x,s) + ^(x,t,s). 

Now writing (l) in the equivalent form 

W(x,t) = /G(x,x* jt, )dx’ 

+ /G(x,x»;t, s)t(x , ,s,0)dx 1 + i|f (x, t, s) 


and using the semigroup property 


/G(x,x T j t, 0)p (x ! )dx J = / G (x, x" 


a _ 

3 j 


;"/G(x",x , ;s,0)p(x , )dx* 


and the continuity w.p.l. of t(x, t, s) gives (2). Q.E.D. 

The concern of the paper is restricted to systems with 
controls, which are linear in W(x, t) and which appear linearly 
in the differential equation. Thus adding a control term to (3) 
we have, formally, 
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dW = j£W(x,t)*dt + cr(x,t)dz + b(x, t)u(x,t) (3a) 


where the control is 


u(x,t) = /W(y,t)k(y,x,t)dy (4) 

Theorem 3.2 gives meaning to (3a). 


Theorem 3.2, Suppose (E-6) and all the assumptions of 
Theorem 3.1, and let u(x,t) be given by (4). Then there is a 
Holder continuous (w.p.l) version of (5) 


W(x,t) = /G(XjX’ * t, 0)cp (x 1 )dx* + / dz* /G(x,x* ;t, t’ )cr(x' } t* )dx’ 

o 

t (5 > 

+ / dt* /G(x,x* * t, t* )b(x’ } t' )u(x’ ,t' )dx’ 
o 


Furthermore (w.p.l.) D.W(x^t) and D.D.W(x,t) are Holder con- 
tinuous on R, and both W(x,t) and jCW(x^t) tend to zero as 
x -* 2 d. 

There is a kernel B(x,x* ;t,t ’ ) } so that 


W(x,t) = / dt* /B(x,x* Jt, t' )q(x* } t* )dx* (6) 

o 


where q(x, t) is the sum of the first two terms on the right side 



1 6 


of (5). B(x,x';t,t*) maps f C° +a (R) into C° +q; (R). W(x,t) has 

the Ito differential 

dW(x,t) = £W(x,t)dt + cr(x,t)dz + b(x, t)/k(y,x, t)W(y,t)dydt (7) 

Proof . Let W(x, t) be Holder continuous on R (exponent 
a) and tend to zero as x -» cD. Let the Holder exponent in (E- 6 ) 
be y ^ a. Then 

f(x’ ,t* ) = b(x» ,t‘ )/W(y,t» )k(y,x' f t* )dy ( 8 ) 

is Holder continuous on R (exponent a) and tends to zero as 
x* -» cD. Thus, (by Lemma 2.3A) the last term in (5) maps 
W(x,t) e continuously into Hence (5) also is a con- 

— O Q 

tinuous linear map of into C_ . 

2 +a 2+a 

Write (5) as 

t 

W(x,t) = q(x,t) + / /M(x,x* ;t, t*)W(x' ,V )dx’ ( 9 ) 

o 

where 

M(x,x';t,t' ) - /G(x,y;t,t f )b(y,t* )k(x',y,t)dy. 

The kernel M(x,x* jt,t* ) must also correspond to a continuous 
linear map of C° +Q; into 0°^. By Theorem 3.1, and Lemma 2.3A 

+" ~~ — o 

See Lemma 2.3A for the definition of ^ 2 +a 
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q(x,t) is in C 2+q; for some a > 0 w.p.l. Then, the theory of 
Volterra integral equations asserts the existence of a 
W(x,t) e C^ +a satisfying ( 9 ) (and, hence, (5)) w.p.l. It also 
yields the representation ( 6 ) 

A 

The assertion concerning the Ito differential follows 
exactly as in the proof of Theorem 3.1. Q.E.D. 



4. The Solution to the Optimum Control Problem 


The solution is divided into four Theorems. Theorem 4.1 
establishes some required properties of a partial differential in- 
tegral equation (the analog of the Ricatti equation). Theorem 4.2 
establishes a formula for the cost corresponding to a fixed control. 
Then^ (Theorem 4.3) the usual dynamic programming technique of 
quasilinearization (or approximation in policy space) is applied 
to obtain a sequence of costs (and improved controls) which 
(Theorem 4.4) converges to the minimum cost (and optimal control). 

The adjoint of £ } operating on functions of x } is_ written 
as 


* x £( x ) 




- Z D i [d j .(x,t)|(x)] 
+ c(x,t)|(x). 


A,q A 

Define the Banach space of functions on R - 

2+a 

^ 

D x D X[0, T] satisfying the condition that f(x, y, t), ;£ x f(x,y, t) 

•X- 

and £ f(x,y,t) -» 0 as x -> c3D or y ^ or t -> T, and with 

J 


IM 2 + a= W a + ?» D x. f “c. + ?HVlla 

1 1 


+ Z ||d d f}| + Z ||d d f|| + || f || 

,• • x. y. "a . x. y. 'a 11 t"a 

i,0 ii i,0 ii 


norm 
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where 



sup Jf(x,y,t)| + sup^ aj 2 

x,y,teR x,y,teR ^ | x-x» | + 1 y-y’ | u +| t-t* | ^ 

x> ,y* , t’ eR 


Theorem 4.1. Assume the conditions of Theorem 3-2, and 
(E-7). Let Q(x,y, t) he symmetric , Holder continuous on 
D X D x [ 0,T ] and positive definite^ for t e [ 0, T] . Let 
Q(x,y,t) -» 0 a£ x 30 or y 2D. Wr ite 


R t (x,y,t) + (£*+£* )-R(x,y,t) 

+ /b(v,t)[k(x,v,t)R(v,y,t) + k(y,v,t)R(x,v,t)]dv (10) 

= -Q(x,y,t). 


There is a unique symmetric (in x,y) and continuous solution to 
(10) which, in addition, is in C,, . 


Proof. By (E-7), the adjoint operator and its Greens func- 
tion are defined. The proof involves some standard calculations, 
similar to those of Sections 3-5, Chapter 1, [5 ], and most of the 
details are left to the reader. Consider first the adjoint equa- 
tion (11), defined in D X D X [0, T] 


T By positive definiteness, we mean /Q(x,y,t)cp(x)cp(y)dxdy § m/|cp(x)[ dx 
The condition is not actually needed until Theorem 4.4. 
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K t (x,y,t) + (£*+£*)R(x,y,t) = -Q(x,y,t) (11) 

with boundary conditions R(x,y,t) -»0 as x -> 3D or y ->dD 
or t -» T. The unique solution to (ll) can be verified to be the 
symmetric function. 

T 

R(x,y,t) = / ds//dx'dy'G(x' ,x;s,t)G(y' ,yjs,t)Q,(x» ,y' ,s) 
t 

( 12 ) 

T 

= / ds//dx' dy’ G*(xjX’ • t, s)G*(y,y’ j t, s)Q(x’ ,y' , s) . 
t 

Write (12) as 

T 

R(x,y,t) = / /dsdx'G (x,x * ; t, s)h(y,x» ; t, s) 

"b 

where 

h(y,x';t,s) = /G*(y,y»;t,s)Q(x' ,y* ,s)dy* . 


h(y,x* *t,s) is uniformly Holder continuous in y^x* ,t,s for 
s § t, and h(y,x’ *t, s) -> 0 as x -> c® or y -» cffl. Let t S t 
and consider 


Consider the differential equation (*) P = AP + PA 1 - Q, with 
boundary condition P(T) = 0. Let $(t,s) be the fundamental 
matrix of x = Ax. Then (*) has the solution (**) P(t) = 

T 

/ C>(t^ s)Q(s)$' (t, s)ds . Note the similarity in form between (*-*) 
t 

and (12). (Here 1 denotes transpose.) 

Recall that G*(x,x' ;t, t' ) + i # G(x,x* ;t,t' ) = 0 for t < t' . 

t X 



21 


T 

R(x,y,t|t Q ) = / /dsdx«G*(x,x';t,s)h(y,x«;t o ,s) 
t 

_ v 

R(x y,t[ t ) is the solution to the adjoint equation U + £ U = h (with 
t = t in h) to which Lemma 2.3A is applicable.* Thus, for each fixed y, 
$(x,y,t|t ) dhd si*R(x,y,t|t ) tend to zero as x -4 or t -4 T by 

OXO j 

virtue of the properties of Q(x,y,t); also R(x,y,t|t ) is symmetric 

in x,y. Since R(x,y, tj t Q ) and £*R(x,y,t |t Q ) tend to zero as 

x cB or y -> cl), and t is arbitrary, we conclude that the terms 

R(x,y,t|t) = R(x,y,t) and £ R(x,y,t) and :£*R(x,y t) are Holder con- 

x y 

tinuous and tend to zero as x -» cj) or y -> cB or t -> T. Now, to 

complete the proof that R(x,y, t) e we need only show that 

R t (x, y, t) is Holder continuous. But this is true since R(x,y, t|t ) 

has Holder continuous derivatives with respect to t and t in t ^ t § 0 

o o 

(uniformly in x,y in D X d). 

For the rest of the proof, write (10) as the Volterra integral equation 


R(x,y,t) = R(x,y, t) + 

T 

/ ds//dx'dy , G*(x,x';t, s)G*(y,y‘;t, s)M(x«,y' , s) 
t 


(13) 


where M(x',y' ,s) is the integral term in the middle of (10) (with x',y’,s 
substituted for x, y, t, resp.). If there is a solution of the desired form 
to ((14) is obtained by changing the order of integration in ( 13 )) 

T 

R( x ,y,l) = R( x ,y,t) + / ds//R(v,w, s)K(x,yJv,w;t,s)dvdw 

t 

(14) 

T 

+ / ds//R(v,w, s)K(y,x;w,v;t, s)dvdw, 
t 


* The boundary conditions are U(x,t) = 0 on L x {T} + ci Dx [0, T], If the 
time parameter is reversed (changing the terminal manifold D X {T} to an 
initial manifold D X {0}) then, since £* satisfies (El) - (E3), Lemma 2. 3A 
is applicable. 
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where 


K^y; v j w j t, s) = b(v, s)G*(y,w;t, s)/G*(x,x» ; t, s)k(x» ,v, s)dx' , 

then there is a solution of the desired form to (13). It can be 
verified that under the imposed conditions, the kernel K represents"*" 

A A 

a continuous linear map of C° into . 

2+a 2+a 

Finally, it can be verified, via the theory of Volterra equa- 
tions, that (14) does have a unique solution of the desired form. 

Q.E.D. 

Let the control u be given by 

u(x, s) = /k u (v, X, s)W(v, s)dv. 


The the cost becomes 


(cp ; t ) = Euf T dxdydsW (x, s )W (y, s )Q U (x, y, s ) 
9 t 


where 

^ U (x,y, s) = S(x,y,s) + /k U (x,v,s)k U (y,v, s)P(v, s)dv. 


Recall that the system that we are concerned with is defined by 

A 

Theorem 3*2 and has the Ito differential 


"*"The map is given by the sum of the two integrals of (l4). 



23 


dW(Xjt) = ;OT(x, t)dt + cr(x, t)dz + b (x, t)u(x,t)dt, 


and boundary condition W(x, t) -jO, as x -» cD. 

Theorem 4.2. Assume the conditions of Theorem 4.1. Let 
W(x,t) satisfy (5), with the control u(x, t) given by (4), and 
let the initial condition ( given at time t e [0, T]) W(x, t) = 

<p(x) satisfy the conditions on cp(x) of Theorem 3.1. 

Suppose that S(x,y, t),P(v, t),k u (x, v, t) and. b(v,t) are 
Holder continuous in their arguments and S(x,y,t) and k u (x,v,t) 
lend to zero as x -> c£) or y -> cto. Let P(x,t) >0 on D x [0,T] 
anl let S(x,y, t) be symmetric and non-negative definite on 
D x D for each t in [0, T], Then 


T 

C U (cp,t) = F?ff dxdydsQ U (x,y, s)W(x, s)W(y, s) 
v t 


= P(l) + /dxdyR U (x,y,t)<p(x)<p(y). 


( 15 ) 


where 

T 

P(l) = iff dxdydsR u (x,y,s)tr(x, s)tr(y, s), 
t 

R ( x ,y, s ) i£ the function introduced in Theorem 4.1, corresponding to 
Q(x,y, s) = Q U (x,y, s). 

Proof. The assumptions on S(x,y, t), P(v, t), k U (x,v,t) and 
guarantee that Q (x,y, t) satisfies the conditions 
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on Q(x, y, t) in Theorem 4.1. For fixed x,y, the function 
R U ( x ,y, r )W(x,r)¥(y,r) = F(x,y,r) 

has the Ito differential (Theorem 3.2) in [0, T] (we use a version 
of ¥(x, t) for which £¥(x, t) is continuous w.p.l. - see Theorem 
3.2) 

dF(x, y,r) = R^(x,y, r)¥(x, r)¥(y, r)dr + R U (x,y, r) [d¥(x, r)¥(y,r) 

+ ¥(x,r)d¥(y,r) + d¥(x, r)d¥(y, r)] 

= R^(x,y,r)w(x,r)¥(y,r)dr 

+ R U (x,y, r)¥(y, r)[;CW(x, r)dr + cr(x,r)dz + 

+ b(x,r) (/W(v,r)k(v,x,r)dv)dr] 

+ R U (x,y,r)¥(x,r)[;£¥(y,r)dr + cr(y,r)dz + 

+ b(y,r)(/¥(v,r)k(v,y,r)dv)dr] 

+ ^R(x,y,r)a(x,r)a(y,r)dr. 

This, together with R U (x, y, T) = 0 implies (w.p.l for each x,y, t) 
that 

T 

- F(x,y,t) = / dF(x,y,r). 
t 


Furthermore, 
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-E^F(x,y,t) =-EV(x,y,t)¥(x,t)¥(y,t) (l6) 

T 

= E <£/ [ R r^ y ^ r ) W ( x J r ) W ^ r ) 

*fc 

+ R U (x,y,r)¥(y,rX;£¥(x,r)dr + b(x, r)/¥(v, r)k(v, x, r)dv) 

+ R U (x,y,r)W(x,rX£W(y,r)dr + b(y, r)/¥(v, r)k(v, y, r)dv) ] 

T 

+ if R U (x, y,r)a(x ; r) 0 (y J r)dr. 
t 

Each of the integrands on the right side of (l 6) can he defined 
be defined to be a measurable function of (D,x,y,r and absolutely in- 
tegrable over fi x D X D X [t,T]. Thus (l6), together with Fubini»s 
Theorem, implies that 

-E^/dxdy F(x,y,t) =-/ R^(x,y, t)cp(x)cp(y)dxdy = (17) 

T 

= E mJ7 dxdydr[R^(x,y,r)¥(x,r)¥(y,r) 

^ t 

+ R U (x,y,r)¥(y,r)^w(x, r) + b(x,r)/¥(v,r)k(v,x,r)dv) 

+ R U (x,y,r)¥(x,r)(£¥(y,r) + b(y,r)/¥(v,r)k(v,y,r)dv)] 

+ p(t). 

Now ¥(x,t) (w.p.l.) and R u (x,y, t) (for each fixed y e D) 

are continuous and have uniformly continuous first and second 
derivatives, with respect to the x^, in the domain R. Also ¥(x, t) 
(w.p.l.) and R U (x,y,t) tend to zero as x -> Bd. Thus, upon partially 
integrating and using Greens identity to eliminate the boundary in- 
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integrals which are obtained (which are zero, owing to the first two 
sentences of this paragraph), we get (for co not in some null set) 

. T 

// dxdydr R U (x, y, r)W(y, r)£W"(x, r) 
t 

T 

= // dxdydr £ R u (x,y, r)W(y, r)W(x r) . 
t x 


Substituting this in (17), and using the symmetry of R(x,y, r), yields, 
after another change in the order of integration 

-/ R U (x, y, t)cp (x)cp (y ) dxdy = 

T 

= eJJ// dxdydr W(x,r)w(y,r)[R^(x,y,r) + (£*+£* )R U (x,y,r) 

"b 

+ / b(x' ,r)k(x,x* ,r)R u (x’ ,y,r)dx T + (18) 

/ b(x',r)k(y,x',r)R U (x,x«,r)dx«] + p(t). 

Finally, using the relation (10) in (18) yields (15). Q.E.D. 

Write C U (VJ(x, t),t) for the function C U (cp,t) with W(x, t) 
substituted for cp(t). Write d V C U (W(x, t), t) for the Ito differ- 
ential of the cost (corresponding to control u) but where, in the 
expression for dW(x, t), a control v(x, t) replaces the control 
u(x,t). 

Theorem 4.5. Let u(x, t) be a given control (then k U (x,y, t) is 
given ) and assume the other conditions of Theorem 4.2. Let u(x,t) be 
the function v(x,t) which minimizes . 
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T T 

E^ / d V C U (W(x, s),s) + / /dxdydsS(x,y, s)W(x, s)W(y, s) 


9 


+ E V / /dxdsP(x, s)v^(x, s) 


( 19 ) 


9 


or, equivalently, which minimizes 


E^/dxdy R U (x,y, s)[W(y, s)b(x, s)v(x, s) + W(x,s)b(y,s)v(y,s)] 

p 

+ E^/dxP(x, s)v (x,s). 


( 20 ) 


Then 

C U (cp,t) C U (cp,t) (21) 

u(x,s) is given by 

u(x,s) = -b(x,s)/ B_(SZ^I W (y, s)dy (22) 

and 


k U (g,x,s) = -b(x,s)R U (x,g, s)/P(x, s) (2$) 


and the corresponding R u (x, y, t) satisfies the conditions on the 
R U (x,y, t) of Theorem 4,2. Also 
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/ R U (x,y,t)cp(x)cp(y)dxdy g / R U (x,y, t)<p (x)cp (y)dxdy (24) 


for &ny bounded and measurable function cp (x) . 

Proof^. By Theorem 4.2, (19) is non-negative and equals zero 
when v(x,t) = u(x,t). Then any minimizing v(x,t), (provided that 
the corresponding integrals of (19) exist) must leave (19) non- 
positive. This, together with the facts that C V (cp,t) is the sum 
of the last two integrals in (19) and that the first integral of (19) 
equals -C U (cp,t), implies (21). The v(x,t) minimizing (19) is the 
v(x, t) which minimizes 


E 9 [ / dxd y ds [ R g( x ^y^ s ) w ( x , s )W(y,s) + H U (x,y,s)W(y, s ) (*W(x,s)+b(x,s)v(x,s)) 


+ R U (x, y, s ) W (x, s ) (£Vf (y, s)+b (y, s ) v (y, s ) ) ] 


T T 

+ E^ f /dxdydsS(x,y, s)W(x, s)W(y, s) + / /dxdsP(x, s)v 2 (x, s) 


equivalently, which minimizes (20). The v(x,t) minimizing (20) 
is given by (22). The statement below (23) i s easily established 
(via Theorem 4.2 ) since the k U (y,x,t) of (23) satisfies the con- 
ditions on k V (y,x,t) in the Hypothesis of Theorem 4.2. (24) is 

valid for all doubly differential functions cp(x) which are zero 
on ci) since 
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T 

C V (cp,t) = /dxdyR V (x 3 y,t)cp(x)cp(y) + \ ff dxdydrR V (x ,, y, s)cr(x, s)o(y, s) 

t 

T 

g /dxdyR U (x ; y, t)cp (x)cp(y) + \ ff dxdydsR U (x,y, s)a(x, s)a(y, s) 

t 

for all such cp(x). Hence (24) is valid for all functions which are 
(almost everywhere) pointwise limits of a bounded sequence of such 
qp(x). Q.E.D. 

Theorem 4.4 is the optimality theorem. Let k n , Q n and R n 
u u u 

correspond to k } Q and R , resp. 

Theorem 4.4. Let u q (x, t) be given and let the correspond - 
ing k°(y,x, t) satisfy the conditions on k U (y,x, t) in Theorem 
4.2. Suppose the other conditions of Theorem 4.2 hold. Define 

U n (x,t) from u^ ^(xjt), n = 1,..., via the procedure of Theorem 4.3. 
Then R n (x,y,t) converges pointwise ( almost everywhere ) to an R(x ; y^t) 
which satisfies the conditions of Theorem 4.1. The control u(x^t) 
corresponding to R(x,y,t) via (2p) (see (22)) 


U(X,S) * 


= /k(y,x,s)W(y,s)dy 


( 35 ) 


is optimal in that C U (cp,t) g C v (cp^t) for any other control 
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v(x,s) = /k V (y, x, s)w(y, s)dy (25 a) 

where k V (y, x, t) satisfies the condition on the k U (y, x, t) in 
Theorem 4.2. R(x,y, t) also satisfies the boundary conditions on 
the R U (x,y,t) of Theorem 4.2, and the ' Ricatti * equation 

R t (x,y,t) + (/ + /)R(x,y,t) (26) 

+ /b(v,t)[k(x,v,t)R(v,y,t) + k(y, v, t)R(x, v,t)]dv 
= -Q(x,y,t) 

where k(y,x,t) is given by (25) k(x,v,t) = -b(v,t)R(v, x, t)/P(v, t) 
and also 

Q(x,y,t) = s(x,y,t) + /k(x,v,t)k(y,v,t)P(v,s)dv. (27) 

Proof. The proof is divided into three steps. First, we 
show that R n (x, y, t) converges (almost everywhere) to some function 
R(x,y, t),* second that R(x,y, t) is smooth and satisfies (26), and 
third, that R(x, y, t) corresponds to the optimal control. By (24) 
(where u and u are replaced by u^ and u n+ j_) and the non-negative 
definiteness of the R 1 (x,y,t), 

/dxdy[R n (x,y, t)-R n+1 (x,y,t)]cp(x)cp(y) g 0 (28) 

for any bounded measurable cp(x). Also the R n (x, y, t) 


are continuous 
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in DX DX [0, T] . Hence, the diagonal values R n (x,x, t) are non- 
negative and non- increasing as n increases, and R n (x,x, t) J, R(x,x, t) 
(almost everywhere) for some function R(x,x, t). This together with 

max | R n (x,y, t) | § max | R n l (x, x, t)|, implies that the R n (x,y, t) are 
x, y x 

uniformly bounded. In fact, the pointwise convergence implies that 
the diagonal values converge 'almost uniformly' in the sense that 
for any fixed e > 0 there is a function a(N, e) tending to zero 
as N co and a set S^C D with Lebesgue measure a(N, e) so that, 
for any m, n > N 

0 g R n (x,x, t) - R m (x,x, t) < e (29) 

on D - S . Next, suppose that, for some m > n > N 

R m (x',x",t) - R n (x',x",t) > 2c, (29a) 

on some (x f ,x" ) e (D-S ) X (D-S^). Then, by continuity and symmetry, 
there are neighborhoods A', A" of x' ,x", resp (A', A" are assumed to 
be in D-S^) so that (29a) and (29) hold on A' x A" U A" X A* . Let 
l(A)(x)be the characteristic function of the set A. Set cp(x) = 
l(A' U A")(x). Then using this and the diagonal (29) bound in (28) 
gives 

/[R n (x, y, t ) -R m (x, y, t) ] I (A 1 UA" ) (x) • I (A' UA" ) (y) dxdy 


g (2e-4e)|i(A* )n(A") < 0 
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(u(*) is Lebesgue measure on D, a contradiction. Since a can be 
made arbitrarily small by increasing N, we conclude that R n (x, y, t) 
converges almost everywhere to a function R(x,y,t). Furthermore, it 
is clear that R(x, y, t) is symmetric, measurable and bounded almost 
everywhere by r(x,y,t), where r(x,y,t) is some function which tends 
to zero as x cfc or y -» cD. 

To continue, we use the representation (see Theorems 4.1 and 
4.3 for terminology) 


■r.n+1, , x ~n+l, .. 

R (x,y,t) = R (x,y,t) + 


T 


+ / /dsdvdwK n+ (x,y* v,w* t, s)R n (v,w, s) 
t 


(30) 


T 


+ J / dsdvdwK n+ ' 1 '(y, x*w,v;t, s)R n (v,w, s) 
t 


rn+1 , 


R^O^y,*) = / /dsdx'dy*G*(x,x»;t,s)G*(y,y>;t,s)Q n+1 (x-,y',s) (31) 
Q n+1 (x»,y> ,s) = S(x‘,y* , s) + /k™ 1 (x' , v, s)k n+1 (y« , v, s)P(v, s)dv (32) 


k n+1 (x',v,s) = -b(v,s)R n (v,x',s)/p(v,s) 


(33) 


2 

R nfl (x,y;v,w;t,s) - - ~ b p (y, t , s ) / g* (x, x « ; t , s ) R n (■ v, x » , s ) dx 1 » . (34) 


The left side of (30) tends (almost everywhere) to R(x,y, t). 
Similarly (and we omit the uninteresting details) the limit of each 
sequence of integrals can be replaced by the integral of the (almost 
everywhere) limit of the integrands. 
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Thus, a. e. 

T 

R(x,y,t) = R(x,y,t) + / /dsdvdwK(x,y; v,w; t, s)R(v,w, s) 

t 

(35) 

T 

+ / /dsdvdwK(y,x;w,v;t, s)R(v,w, s), 
t 


where the kernel K is given hy (34) with R(v,x’,s) replacing 
R n (v,x',s). 

Consider the R(x,y,t) term in (35)- Since Q,(x,y, t) is 
symmetric, hounded, measurable and tends to zero (almost everywhere) 
as x -> c£) or y -» c®, R(x,y,t) is Holder continuous and tends to 
zero as x -» cD or y -» 3 d or t -> T. If a member of this latter 
class is substituted for the Q(x,y,t), then R(x,y,t) e C° 2+Q ,- 
Similarly the map K (the sum of the integrals in (35)) takes 
R(x,y, s) into a Holder continuous function which tends to zero as 
x -> 3 d, y ->3 d or t -» T. If a member of this class is substituted 
for the R(x,y, s) in the kernel K, then the sum of the integrals in 
(35) is in Cg . (Recall the identical assertion in the proof of 
Theorem 4.1.) These considerations imply that R(x,y,t) is indeed 
n° 

in C 0 . 

2+a 

Upon differentiating (35) and using G (x,x*;t, s) = 

-^*G(x,x’ • t, s) we get (26). 

Now, note that the u(x,t) in (25) is the v(x,t) which 
minimizes (19) and (20). Thus, letting v(x, t) be a control of the 
form (25a), (19) yields 


/ d U C U (W(x,s),s) 
"t 


- T - 

g / dV(W(x,s),s) 
9 t 


+ / /dsdxdyS(x,y, s)W(x, s)W(y, s) 

T 

+ / /dsdxP(x, s)u 2 (x, s) 

"b 


+ 


v 

<P 


T 

/ /dsdxdyS(x,y, s)W(x, s)w(y, s) 
t 


— T 

V o 

+ E / / dsdxP(x, s)v (x, s). 

^ t 


Since the first terms on the left and right of (3 6 ) are equal, 
implies C U (cp,t) § C V (cp,t). Q.E.D. 
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